Fingerprint anti-counterfeiting method and electronic device

ABSTRACT

A fingerprint anti-counterfeiting method and an electronic device are provided. The fingerprint anti-counterfeiting method includes: After detecting a fingerprint input action of a user, an electronic device obtains a fingerprint image generated by the fingerprint input action, and obtains a vibration-sound signal generated by the fingerprint input action. The device determines, based on a fingerprint anti-counterfeiting model, whether the fingerprint input action is performed by a true finger. The fingerprint anti-counterfeiting model is a multi-dimensional network model obtained through learning based on fingerprint images for training and corresponding vibration-sound signals. The fingerprint anti-counterfeiting method in embodiments of this application helps improve a protection capability of the electronic device for a fake fingerprint attack.

This application is a National Stage of International Application No.PCT/CN2020/135111, filed on Dec. 10, 2020, which claims priority toChinese Patent Application No. 201911300678.0, filed on Dec. 17, 2019,both of which are hereby incorporated by reference in their entireties.

TECHNICAL FIELD

This application relates to the field of computer technologies, and inparticular, to a fingerprint anti-counterfeiting method and anelectronic device.

BACKGROUND

Human fingerprints are different from each other, and are commonbiometric features for identity recognition. Currently, the fingerprintshave been widely applied to fields such as device unlocking, an accesscontrol system, an attendance system, and financial payment. However, indaily life, fingerprint information is likely to be leaked andcounterfeited. This brings a security risk to an identity authenticationsystem based on fingerprint recognition. In addition, as intelligentterminals such as mobile phones enter a bezel-less screen era, anunder-display optical fingerprint recognition solution has been afingerprint recognition technology preferred by many vendors. Withmaturity of an under-LCD optical fingerprint recognition technology, theunder-display optical fingerprint recognition solution has become atrend. Specifically, in the under-display optical fingerprintrecognition technology, an under-display light source is used to emitlight to a finger surface, and reflected light is converted, by using aphotosensitive component, into an electrical signal includingfingerprint information, to further complete fingerprint informationrecognition. However, because light can reach only an epidermis layer offinger skin but cannot reach a dermis layer, an optical fingerprintrecognition device may be deceived by a fake fingerprint (especially a3D fingerprint film) that is easy to produce and has low costs.

In conclusion, a fake fingerprint attack is a pain point in an existingfingerprint recognition technology, and causes extensive impact on userprivacy and property security. Currently, solutions to the fakefingerprint attack may be summarized into two types. One type issolutions based on hardware. An additional hardware module is added todetect biometric feature signals of a dermis layer, a blood vessel, aheart rate, and blood oxygen of a finger, and detect a fake fingerprintattack in combination with fingerprint recognition. The other type issolutions based on software algorithms Based on features of true andfake fingerprints, an image recognition technology is used to determinewhether a fingerprint image is true or fake.

In terms of the hardware-based solutions to the fake fingerprint attack,the industry proposes a solution that uses a combination of afingerprint sensor and a heart rate sensor to recognize whether afingerprint is true or fake. The heart rate sensor detects a heart ratebased on a difference between blood content in soft tissue duringsystole and diastole, to further determine whether a fingerprint imageis from a living object. For example, a vendor discloses its single-chipsolution to live fingerprint detection. The solution uses an integratedhardware sensor to recognize a blood flow and a heart rate of a finger,to detect a fake fingerprint attack.

In addition, a solution that uses an optical coherence tomographytechnology and an optical microvascular angiography technology todetermine whether a fingerprint is true or fake is also reported inrelated literature. In this solution, information about structures suchas an epidermis layer, a dermis layer, and sweat glands of a fingertipis obtained by using an optical coherence tomography apparatus, and amicrovascular structure and blood flow information are obtained by usingan optical microvascular angiography apparatus, to further performliveness detection based on the two types of obtained information. Inaddition, a disclosed patent shows that an infrared hardware module maybe added to perform liveness detection by detecting various substancesunder a dermis layer of a finger based on an infrared reflectivity.

However, the hardware-based solutions to the fake fingerprint attackhave the following disadvantages: First, an additional feature detectionhardware module increases device costs. For example, costs of an opticalcoherence tomography apparatus are as high as hundreds of thousands ofdollars, and there is a low probability that the apparatus is used in amobile phone. Second, it is difficult to integrate an additional featuredetection hardware module with a terminal device. Third, fakefingerprint attack methods emerge one after another. In thehardware-based solutions to the fake fingerprint attack, upgrading isdifficult in the face of emerging new attack methods.

In terms of the software algorithm-based solutions to the fakefingerprint attack, related patents propose fake fingerprint detectionsolutions based on SVM and sparse representation and based on a naiveBayes classifier. In the two solutions, based on different conventionalmachine learning methods, true fingerprint and fake fingerprint imagedatasets are collected, feature extraction and learning are performed,and finally an obtained model is used to determine whether a fingerprintis true or fake. In addition, a related patent discloses a method fordetermining, by using a convolutional neural network, whether afingerprint is true or fake. The neural network is trained by using atrue fingerprint dataset and a fake fingerprint dataset. In afingerprint recognition process, a convolutional neural network obtainedthrough training is used to classify fingerprints into true or fake.

However, the solutions based on the software algorithms also have thefollowing disadvantages: For example, a true fingerprint image and afake fingerprint image have similar features, which are difficult todetermine by human eyes. Conventional machine learning solutions lackin-depth mining of fingerprint information such as fingerprint texturefeatures. A current deep learning solution is still limited on asmall-area fingerprint sensor of a mobile device such as a mobile phone,and in particular, a 3D fake fingerprint attack recognition rate needsto be further improved.

SUMMARY

To reduce impact of an existing fake fingerprint attack on fingerprintrecognition reliability, this application provides a fingerprintanti-counterfeiting method and an electronic device.

An embodiment of this application provides a fingerprintanti-counterfeiting method, including the following steps:

obtaining a fingerprint image generated by a fingerprint input action;

obtaining a vibration-sound signal generated by the fingerprint inputaction; and

determining, based on a fingerprint anti-counterfeiting model, whetherthe fingerprint image and the vibration-sound signal are generated by afingerprint input action of a true finger, where the fingerprintanti-counterfeiting model is a multi-dimensional network model obtainedthrough fusion learning or separate learning based on a plurality offingerprint images for training and corresponding vibration-soundsignals in a multi-dimensional anti-counterfeiting network.

In an embodiment, the fingerprint anti-counterfeiting model includes amulti-dimensional network model obtained through training based on aplurality of training sample pairs including the plurality offingerprint images for training and the corresponding vibration-soundsignals in the multi-dimensional anti-counterfeiting network. The stepof determining, based on a fingerprint anti-counterfeiting model,whether the fingerprint input is generated by a fingerprint input actionof a true finger includes:

forming a to-be-determined sample pair by using the fingerprint imageand the vibration-sound signal; and

inputting the to-be-determined sample pair into the multi-dimensionalnetwork model to obtain a computation result.

In an embodiment, the multi-dimensional anti-counterfeiting networkincludes a fingerprint image subnetwork and a vibration-sound signalsubnetwork, respectively for performing feature extraction on thefingerprint image and the vibration-sound signal.

In an embodiment, the plurality of fingerprint images for training andthe corresponding vibration-sound signals are separately normalizedbefore the plurality of fingerprint images for training and thecorresponding vibration-sound signals form the plurality of trainingsamples. The method further includes a step of separately normalizingthe fingerprint image and the vibration-sound signal before the step offorming a to-be-determined sample pair by using the fingerprint imageand the vibration-sound signal.

In an embodiment, an electronic device stores the fingerprintanti-counterfeiting model. The multi-dimensional anti-counterfeitingnetwork includes a fingerprint image subnetwork and a vibration-soundsignal subnetwork. The multi-dimensional anti-counterfeiting network istrained based on a training set including the plurality of fingerprintimages for training and the plurality of corresponding vibration-soundsignals. The step of determining, based on a fingerprintanti-counterfeiting model, whether the fingerprint image and thevibration-sound signal are generated by a fingerprint input action of atrue finger includes:

inputting the fingerprint image into the fingerprint image subnetwork toobtain a first feature vector;

inputting the vibration-sound signal into the vibration-sound signalsubnetwork to obtain a second feature vector;

fusing the first feature vector and the second feature vector to obtaina third feature vector;

classifying, by the electronic device, the third feature vector tocalculate a classification result; and

determining, by the electronic device based on the classificationresult, that the fingerprint input action is generated by the truefinger or generated by a fake finger.

In an embodiment, an electronic device stores the fingerprintanti-counterfeiting model. The multi-dimensional anti-counterfeitingnetwork includes a fingerprint image subnetwork and a vibration-soundsignal subnetwork. The fingerprint image subnetwork is trained based ona fingerprint image training set including the plurality of fingerprintimages for training, and the vibration-sound signal subnetwork istrained based on a vibration-sound signal training set including theplurality of vibration-sound signals corresponding to the plurality offingerprint images for training. The step of the determining, based on afingerprint anti-counterfeiting model, whether the fingerprint image andthe vibration-sound signal are generated by a fingerprint input actionof a true finger includes:

inputting the fingerprint image into the fingerprint image subnetwork toobtain a first feature vector;

inputting the vibration-sound signal into the vibration-sound signalsubnetwork to obtain a second feature vector;

classifying, by the electronic device, the first feature vector and thesecond feature vector to calculate a classification result; and

determining, by the electronic device based on the classificationresult, that the fingerprint input action is generated by the truefinger or generated by a fake finger.

In an embodiment, the determining, by the electronic device based on theclassification result, that the fingerprint input action is generated bythe true finger or generated by a fake finger includes:

When the classification result includes a confidence level, if theconfidence level is greater than or equal to a specified threshold, theelectronic device determines that the fingerprint input action isgenerated by the true finger; or if the confidence level is less thanthe specified threshold, the electronic device determines that thefingerprint input action is generated by the fake finger. The confidencelevel is a confidence level that the fingerprint input action isgenerated by the true finger.

In an embodiment, the method further includes:

Before inputting the fingerprint image and the vibration-sound signalinto the multi-dimensional anti-counterfeiting network, the electronicdevice fuses the fingerprint image and the vibration-sound signal; and

after inputting the fingerprint image and the vibration-sound signalinto the multi-dimensional anti-counterfeiting network, the electronicdevice separates the fingerprint image from the vibration-sound signal.

In an embodiment, the fingerprint image subnetwork includes aconvolutional neural network, and the vibration-sound signal subnetworkincludes a recurrent neural network.

In an embodiment, the method further includes: detecting whether atrigger event occurs, and when a detection result is that the triggerevent occurs, controlling a fingerprint sensor to start fingerprintimage collection to obtain the fingerprint image generated by thefingerprint input action, and controlling a vibration-sound sensor tostart vibration and sound collection to obtain the vibration-soundsignal generated by the fingerprint input action.

In an embodiment, the method further includes: detecting whether atrigger event occurs, and when a detection result is that the triggerevent occurs, controlling a fingerprint sensor to start fingerprintimage collection to obtain the fingerprint image generated by thefingerprint input action, controlling a vibration-sound excitationsource to emit a vibration-sound excitation signal, and controlling avibration-sound sensor to start vibration-sound signal collection aftera preset latency that starts to be calculated when the vibration-soundexcitation signal is emitted, to obtain the vibration-sound signalgenerated by the fingerprint input action.

In an embodiment, when the electronic device determines that thefingerprint input action is generated by the fake finger, the methodfurther includes:

The electronic device determines that fingerprint recognition fails, andsends prompt information indicating that the fingerprint recognitionfails.

In an embodiment, when the electronic device determines that thefingerprint input action is generated by the true finger, the methodfurther includes:

The electronic device determines whether the fingerprint image matches apreset fingerprint image.

If the fingerprint image matches the preset fingerprint image, theelectronic device determines that fingerprint recognition succeeds.

In an embodiment, the vibration-sound signal includes a mechanicalvibration signal and a sound signal, and the vibration-sound sensorincludes at least one of a microphone, a sound sensor, an accelerationsensor, a crash sensor, a displacement sensor, the acceleration sensor,and a force sensor.

In an embodiment, the fingerprint sensor includes at least one of anunder-display/in-display optical fingerprint sensor, an ultrasonicfingerprint sensor, and a capacitive fingerprint sensor.

An electronic device is provided, including an input unit, a processorunit, and a storage unit.

The input unit includes a fingerprint sensor and a vibration-soundsensor. The fingerprint sensor is configured to collect a fingerprintimage of a fingerprint input action, and the vibration-sound sensor isconfigured to collect a vibration-sound signal generated by thefingerprint input action.

The storage unit stores a computer program and a fingerprintanti-counterfeiting model. The fingerprint anti-counterfeiting model isa multi-dimensional network model obtained through fusion learning orseparate learning based on a plurality of fingerprint images fortraining and corresponding vibration-sound signals in amulti-dimensional anti-counterfeiting network. The computer programincludes instructions, and when the instructions are executed by theprocessor, the electronic device performs the following steps:

invoking the fingerprint sensor and the vibration-sound sensor tocollect the fingerprint image and the vibration-sound signal; and

determining, based on the collected fingerprint image, the collectedvibration-sound signal, and the fingerprint anti-counterfeiting model,whether the fingerprint input action is generated by a true finger orgenerated by a fake finger.

In an embodiment, the vibration-sound sensor includes a vibration-soundexcitation source and a vibration-sound sensing module. Thevibration-sound excitation source is configured to emit avibration-sound excitation signal, and the vibration-sound sensingmodule is configured to collect the vibration-sound signal generated bythe fingerprint input action.

In an embodiment, the vibration-sound excitation source is configured toemit the vibration-sound excitation signal when being triggered by thefingerprint input action, and the vibration-sound sensing module isconfigured to start, after a preset latency that starts to be calculatedwhen the vibration-sound excitation source is triggered, detecting thevibration-sound signal generated by both the fingerprint input actionand the vibration-sound excitation signal.

In an embodiment, the vibration-sound signal includes a mechanicalvibration signal and a sound signal, and the vibration-sound sensorincludes at least one of a microphone, a sound sensor, an accelerationsensor, a crash sensor, a displacement sensor, the acceleration sensor,and a force sensor.

In an embodiment, the fingerprint sensor includes at least one of anunder-display/in-display optical fingerprint sensor, an ultrasonicfingerprint sensor, and a capacitive fingerprint sensor.

In an embodiment, the fingerprint anti-counterfeiting model includes amodel obtained through training based on a plurality of training samplepairs including the plurality of fingerprint images for training and thecorresponding vibration-sound signals. The processor unit is furtherconfigured to: form a to-be-determined sample pair by using thefingerprint image and the vibration-sound signal, input theto-be-determined sample into the fingerprint anti-counterfeiting modelto obtain a confidence level that the fingerprint input action is a truefingerprint, and determine, based on the confidence level and aspecified threshold, whether the fingerprint input action is generatedby the true finger.

In an embodiment, the multi-dimensional anti-counterfeiting networkincludes a fingerprint image subnetwork and a vibration-sound signalsubnetwork.

In an embodiment, the fingerprint image subnetwork is obtained throughtraining based on a fingerprint image training set including theplurality of fingerprint images for training. The vibration-sound signalsubnetwork is obtained through training based on a vibration-soundsignal training set including the plurality of vibration-sound signalscorresponding to the plurality of fingerprint images for training. Theprocessor unit is further configured to: input the fingerprint imageinto the fingerprint image subnetwork to obtain a first feature vector;input the vibration-sound signal into the vibration-sound subnetwork toobtain a second feature vector; fuse the first feature vector and thesecond feature vector to obtain a third feature vector; classify thethird feature vector to calculate a classification result; anddetermine, based on the classification result, that the fingerprintinput action is generated by the true finger or generated by the fakefinger.

In an embodiment, the fingerprint image subnetwork is obtained throughtraining based on a fingerprint image training set including theplurality of fingerprint images for training. The vibration-sound signalsubnetwork is obtained through training based on a vibration-soundsignal training set including the plurality of vibration-sound signalscorresponding to the plurality of fingerprint images for training. Theprocessor unit is further configured to: input the fingerprint imageinto the fingerprint image subnetwork to obtain a first feature vector;input the vibration-sound signal into the vibration-sound signalsubnetwork to obtain a second feature vector; classify the first secondfeature vector and the second feature vector to calculate aclassification result; and determine, based on the classificationresult, that the fingerprint input action is generated by the truefinger or generated by the fake finger.

In an embodiment, the fingerprint image subnetwork includes aconvolutional neural network, and the vibration-sound signal subnetworkincludes a recurrent neural network.

An electronic apparatus is provided, including a processor and a memory.The memory stores at least one instruction, and when the at least oneinstruction is executed by the processor, the fingerprintanti-counterfeiting method described in any one of the foregoingembodiments can be implemented.

Compared with the conventional technology, in the electronic device, thefingerprint anti-counterfeiting method, and the electronic apparatus inthis application, determining, based on the vibration-sound signal,whether the fingerprint image is true or fake may effectively defendagainst a fake fingerprint attack (especially a 3D fake fingerprintattack) faced by a current fingerprint solution. In addition, astructure and an integrated application of the vibration-sound sensorare also relatively simple, and therefore, a cost increase andintegration difficulty caused by a hardware-based anti-counterfeitingsolution can be effectively avoided. In addition, because the networkmodel (for example, a first multi-dimensional network model and a secondmulti-dimensional network model) used for fingerprintanti-counterfeiting may provide fast upgraded protection againstcontinuously emerging fake fingerprint attack manners, reliability isrelatively high.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a schematic block diagram of a structure of an electronicdevice according to an embodiment of this application;

FIG. 2 is a schematic diagram of a sample pair that includes afingerprint image and a vibration-sound signal and that is obtained bythe electronic device shown in FIG. 1 ;

FIG. 3 is a schematic diagram of a principle of a fingerprintanti-counterfeiting model according to an embodiment;

FIG. 4 is a schematic diagram of a principle of a fingerprintanti-counterfeiting model according to another embodiment;

FIG. 5A and FIG. 5B are a flowchart of a fingerprint anti-counterfeitingmethod according to an embodiment of this application;

FIG. 6 is a schematic block diagram of an electronic apparatus using thefingerprint anti-counterfeiting method shown in FIG. 5A and FIG. 5B; and

FIG. 7 is a schematic diagram of a structure of an electronic apparatusfor performing a fingerprint anti-counterfeiting method according to anembodiment of this application.

DESCRIPTION OF EMBODIMENTS

The following clearly and completely describes the technical solutionsin embodiments of this application with reference to the accompanyingdrawings in embodiments of this application. It is clear that thedescribed embodiments are merely some but not all of embodiments of thisapplication. All other embodiments obtained by persons of ordinary skillin the art based on embodiments of this application without creativeefforts shall fall within the protection scope of this application.

To make the objectives, features and advantages of this application morecomprehensible, the following further describes in detail thisapplication with reference to the accompanying drawings and specificembodiments.

To make persons skilled in the art understand the solutions in thisapplication better, the following clearly and completely describes thetechnical solutions in embodiments of this application with reference tothe accompanying drawings in embodiments of this application. It isclear that the described embodiments are merely some but not all ofembodiments of this application. All other embodiments obtained bypersons of ordinary skill in the art based on embodiments of thisapplication without creative efforts shall fall within the protectionscope of this application.

In the specification, claims, and accompanying drawings of thisapplication, the terms “first”, “second”, “third”, and the like areintended to distinguish between different objects but do not indicate aparticular order. In addition, the term “include” and any other variantthereof are intended to cover a non-exclusive inclusion. For example, aprocess, method, system, product, or device that includes a series ofsteps or units is not limited to the listed steps or units, butoptionally further includes an unlisted step or unit, or optionallyfurther includes another inherent step or unit of the process, method,product, or device.

FIG. 1 is a schematic block diagram of a structure of an electronicdevice 10 according to an embodiment of this application. The electronicdevice 10 may be any mobile or portable electronic device or anotherintelligent terminal device, including but not limited to a mobilephone, a mobile computer, a tablet computer, a personal digitalassistant (PDA), a wearable device, and a combination of two or more ofthe foregoing items; or may be another device having a fingerprintrecognition function, for example, a fingerprint-based access controldevice, a financial device having a fingerprint recognition function, oranother identity authentication device having a fingerprint recognitionfunction.

As shown in FIG. 1 , the electronic device 10 may include componentssuch as an input unit 11, a processor unit 12, an output unit 13, acommunications unit 14, a storage unit 15, and a peripheral interface16. These components perform communication by using one or more buses.Persons skilled in the art may understand that the structure of theelectronic device shown in the figure does not constitute any limitationto this application. The electronic device may have a bus structure or astar structure, or may include more or fewer components than those shownin the figure, or combine some components, or have different componentarrangements.

The input unit 11 is configured to implement interaction between a userand the electronic device 10 and/or input information into theelectronic device 10. In an implementation, the input unit 11 mayreceive digit or character information entered by the user, to generatea signal input related to user setting or function control. The inputunit 11 may further include a touch panel, and the touch panel may beanother human-machine interaction interface, and include elements suchas a physical input button and a microphone, or may be another apparatusfor capturing external information, such as a camera. The touch panel,also referred to as a touchscreen or a touch screen, may collect anoperation action of the user touching or approaching the touch panel,for example, an operation action performed by the user on the touchpanel or at a position close to the touch panel by using any appropriateobject or accessory such as a finger or a stylus, and drive acorresponding connection apparatus according to a preset program.Optionally, the touch panel may include two parts: a touch detectionapparatus and a touch controller. The touch detection apparatus detectsa touch operation of the user, converts the detected touch operationinto an electrical signal, and transmits the electrical signal to thetouch controller. The touch controller receives the electrical signalfrom the touch detection apparatus, converts the electrical signal intocoordinates of a touch point, and then sends the coordinates to aprocessing unit. The touch controller may further receive a command sentby the processing unit and execute the command. In addition, the touchpanel may be implemented in a plurality of types, such as a resistivetype, a capacitive type, an infrared ray, and a surface acoustic wave.In another implementation of this application, the physical input buttonused for the input unit may include but is not limited to one or more ofa physical keyboard, a function button (such as a volume control buttonor an on/off button), a trackball, a mouse, a joystick, and the like.The input unit in a microphone form may collect a voice entered by theuser or an environment, and convert the voice into a command that is inan electrical-signal form and that can be executed by the processingunit.

The processor unit 12 is further used as a control center of theelectronic device, is connected to each part of the entire electronicdevice by using various interfaces and lines, and performs variousfunctions of the electronic device and/or data processing by running orexecuting a software program and/or module stored in the storage unitand invoking data stored in the storage unit. The processor unit mayinclude an integrated circuit (IC for short), for example, the processorunit may include a single packaged IC, or may include a plurality ofconnected packaged ICs that have a same function or different functions.For example, the processor unit may include only a central processingunit (CPU for short), or may be a combination of a GPU, a digital signalprocessor (DSP for short), and a control chip (for example, a basebandchip) in the communications unit. In an implementation of thisapplication, the CPU may be a single computing core, or may include aplurality of computing cores.

The output unit 13 includes but is not limited to an image output unitand a sound output unit. The image output unit is configured to output atext, a picture, and/or a video. The image output unit may include adisplay panel, for example, a display panel configured in a form of anLCD (Liquid Crystal Display), an OLED (Organic Light-Emitting Diode), afield emission display (FED for short), or the like. Alternatively, theimage output unit may include a reflective display, for example, anelectrophoretic display or a display using an interferometric modulationof light technology. The image output unit may include a single displayor a plurality of displays of different sizes. In a specificimplementation of this application, the touch panel used for the inputunit can also serve as the display panel of the output unit. Forexample, after detecting a gesture operation of touching or approachingthe touch panel, the touch panel transmits the gesture operation to theprocessing unit to determine a type of a touch event, and subsequently,the processing unit provides corresponding visual output on the displaypanel according to the type of the touch event. Although in FIG. 1 , theinput unit and the output unit are used as two separate components toimplement input and output functions of the electronic device, in someembodiments, the touch panel and the display panel may be integrated toimplement the input and output functions of the electronic device. Forexample, the image output unit may display various graphical userinterfaces (GUI for short) as virtual control components, including butnot limited to a window, a scrollbar, an icon, and a scrapbook, so thata user performs an operation in a touch manner.

The communications unit 14 is configured to establish a communicationschannel, so that the electronic device 20 connects to a remote server byusing the communications channel and downloads media data from theremote server. The communications unit may include communicationsmodules such as a wireless local area network (wireless LAN for short)module, a Bluetooth module, an NFC module, and a baseband module, and aradio frequency (RF for short) circuit corresponding to thecommunications module. The communications unit is configured to performwireless local area network communication, Bluetooth communication, NFCcommunication, infrared communication, and/or cellular communicationssystem communication, for example, Wideband Code Division MultipleAccess (W-CDMA for short) and/or High Speed Downlink Packet Access(HSDPA for short). The communications module is configured to controlcommunication between components in the electronic device, and maysupport direct memory access.

The communications unit 14 may include the radio frequency circuit, andthe radio frequency circuit may be configured to receive and sendinformation or receive and send a signal in a call process. For example,after downlink information of a base station is received, the downlinkinformation is sent to the processing unit for processing; and inaddition, designed uplink data is sent to the base station. For anotherexample, after information sent by an external NFC device is received,the information is sent to the processing unit for processing, and aprocessing result is sent to the external NFC device. Generally, theradio frequency circuit includes a well-known circuit configured toperform these functions, and includes but is not limited to an antennasystem, a radio frequency transceiver, one or more amplifiers, a tuner,one or more oscillators, a digital signal processor, a codec chip set, asubscriber identity module (SIM) card, a memory, and the like. Inaddition, the radio frequency circuit may further communicate with anetwork and another device through wireless communication. The wirelesscommunication may use any communications standard or protocol, includingbut not limited to a GSM (Global System of Mobile communication, GlobalSystem for Mobile Communications), a GPRS (General Packet RadioService), CDMA (Code Division Multiple Access), WCDMA (Wideband CodeDivision Multiple Access), a high speed uplink packet access (HSUPA)technology, LTE (Long Term Evolution), an email, an SMS (Short MessagingService), and the like.

The storage unit 15 may be configured to store a software program and amodule. The processing unit executes various function applications ofthe electronic device and implements data processing by running thesoftware program and the module stored in the storage unit. The storageunit mainly includes a program storage region and a data storage region.The program storage region may store an operating system, an applicationprogram required by at least one function, such as a sound play programor an image display program. The data storage region may store data(such as audio data or an address book) created according to use of theelectronic device and the like. In a specific implementation of thisapplication, the storage unit may include a volatile memory, such as anonvolatile random access memory (NVRAM for short), a phase changerandom access memory (Phase Change RAM, PRAM for short), or amagnetoresistive random access memory (Magnetoresistive RAM, MRAM forshort), or may include a nonvolatile memory, such as at least onemagnetic disk storage component, an electrically erasable programmableread-only memory (EEPROM for short), or a flash memory component, suchas a NOR flash memory or a NAND flash memory. The nonvolatile memorystores the operating system and the application program that areexecuted by the processing unit. The processing unit loads a runningprogram and data from the nonvolatile memory to memory, and storesdigital content in a massive storage apparatus. The operating systemincludes various components and/or drives that are configured to controland manage regular system tasks, for example, memory management, controlof a storage device, and power management, and facilitate communicationbetween various software and hardware. In an implementation of thisapplication, the operating system may be an Android system of Google, aniOS system developed by Apple, a Windows operating system developed byMicrosoft, or an embedded operating system such as VxWorks.

The software program stored in the storage unit 15 may include anapplication program. The application program may include any applicationinstalled on the electronic device, including but not limited to abrowser, an email, an instant messaging service, word processing,keyboard virtualization, a window widget, encryption, digital copyrightmanagement, voice recognition, voice replication, positioning (forexample, a function provided by a global positioning system), musicplayback, and the like.

The peripheral interface 16 may be an interface for connectingperipheral devices such as a main chassis, a display, and a keyboard.The peripheral interface is a logic circuit (which is a logic component)that connects a CPU, a memory, and a peripheral device of a computer,connects two peripheral devices, or connects two machines by using asystem bus, and is a transfer station for information exchange betweenthe CPU and the outside. The peripheral interface 16 may include aninterface circuit, a connection cable, and the like.

It may be understood that the electronic device 10 further includes apower supply 17, and the power supply 17 is configured to supply powerto different components of the electronic device to maintain running ofthe components. Generally, the power supply may be a built-in battery,for example, a common lithium-ion battery or a nickel metal hydridebattery, or may include an external power supply that directly suppliespower to the electronic device 10, for example, an AC adapter. In someimplementations of this application, the power supply 17 may further bedefined in a wider scope; for example, may further include a powermanagement system, a power charging system, a power failure detectioncircuit, a power converter or inverter, a power status indicator (suchas a light-emitting diode), and any other component related to powergeneration, management, and distribution of the electronic device.

In an implementation, the input unit 11 includes a fingerprint sensor111 and a vibration-sound sensor 112, the fingerprint sensor 111 isconfigured to detect a fingerprint image generated by a fingerprintinput action, and the vibration-sound sensor 112 is configured to detecta vibration-sound signal generated by the fingerprint input action. Theprocessor unit 12 can receive the fingerprint image and thevibration-sound signal, and determine, based on a fingerprintanti-counterfeiting model, the fingerprint image, and thevibration-sound signal, whether the fingerprint input action isgenerated by a fingerprint input action of a true finger. Thefingerprint anti-counterfeiting model is a multi-dimensional networkmodel obtained through fusion learning or separate learning based on aplurality of fingerprint images for training and correspondingvibration-sound signals in a multi-dimensional anti-counterfeitingnetwork. It may be understood that the processor unit 12 may furtheroutput, based on the determining, an identity authentication resultindicating that fingerprint recognition succeeds or fails.

The fingerprint sensor 111 includes at least one of an opticalfingerprint sensor (such as an under-display/in-display opticalfingerprint sensor), an ultrasonic fingerprint sensor, and a capacitivefingerprint sensor. The vibration-sound sensor 112 includes at least oneof a microphone, a sound sensor, an acceleration sensor, a crash sensor,a displacement sensor, the acceleration sensor, and a force sensor. Thevibration-sound sensor 112 may be disposed close to the fingerprintsensor 111, for example, located below or on the periphery of thefingerprint sensor 111. It may be understood that the vibration-soundsignal is generated by the vibration-sound sensor 112 based on thefingerprint input action, and the vibration-sound signal may include amechanical vibration signal and a sound signal. In addition, currently,most electronic devices such as mobile phones are integrated withacceleration sensors, such as pendulous accelerometers, non-pendulousaccelerometers, or MEMS accelerometers. The acceleration sensor may beconfigured to detect mechanical vibration signals of components such asan electronic device body, a PCB, and a screen during a fingerprintinput action of a finger. Commonly used sound signal detection sensorsinclude a capacitive microphone, a piezoelectric microphone, an MEMSmicrophone, and the like. An inherent microphone of the electronicdevice 10 may be configured to detect a sound signal generated duringthe fingerprint input action of the finger. As described above, thevibration-sound sensor 112 may collect, by using the acceleration sensorand a sound sensor such as the microphone that are inherent in theelectronic device 10, the vibration-sound signal including themechanical vibration signal and the sound signal. In this way, atechnical effect of simple integration and application of thevibration-sound sensor 112 is implemented.

Specifically, after collecting the mechanical vibration signal and thesound signal, the vibration-sound sensor 112 may obtain thevibration-sound signal through sampling, where a duration of thevibration-sound signal may be 0.1 s, and a sampling rate may be 22.05Hz, to convert an analog vibration-sound signal into a discretesequential vibration-sound signal.

In an embodiment, the vibration-sound sensor 112 includes avibration-sound excitation source 112 a and a vibration-sound sensingmodule 112 b. The vibration-sound excitation source 112 a is configuredto emit a vibration-sound excitation signal, and after thevibration-sound excitation signal is emitted, the vibration-soundsensing module 112 b is configured to detect a vibration-sound signalgenerated by both a fingerprint input action of a finger and theexcitation signal. In this way, a relatively strong vibration-soundsignal is obtained, and detection sensitivity of the vibration-soundsensor is improved. Specifically, the vibration-sound excitation source112 a may be triggered by the fingerprint input action to emit thevibration-sound excitation signal, and after a preset latency thatstarts to be calculated when the vibration-sound excitation source 112 ais triggered, the vibration-sound sensing module 112 b starts detectingthe vibration-sound signal generated by both the fingerprint inputaction and the excitation signal. It may be understood that thevibration-sound excitation source 112 a includes but is not limited to alinear motor. Generally, the linear motor may generate an excitationsignal whose frequency and amplitude are controllable, and an operatingfrequency of the linear motor may range from 150 Hz to 235 Hz.

Further, the fingerprint image and the corresponding vibration-soundsignal may form a to-be-determined sample pair. For example, theprocessor unit 12 may receive the fingerprint image obtained by thefingerprint sensor and the vibration-sound signal obtained by thevibration-sound sensor, and fuse the fingerprint image and thevibration-sound signal to form a piece of signal data. The foregoingdata fusion manner includes but is not limited to: extracting, from asequence of the vibration-sound signal, a signal that matchesfingerprint image data in time domain, and performing stitching andfusion. It should be noted that, before data fusion of the fingerprintimage and the corresponding vibration-sound signal is performed, thefingerprint image and the corresponding vibration-sound signal should benormalized. For example, a pixel value of the fingerprint image and asampling sequence of the vibration-sound signal are normalized to aspecific range, for example, from 0 to 255. In this way, a datastructure is neat and unified, and data fusion can be further performed.This facilitates data reading and management. Certainly, the fingerprintimage and the vibration-sound signal each may alternatively not benormalized, so that the data includes more original information, and theprocessor unit 12 may perform corresponding preprocessing according to arequirement before the fingerprint image and the vibration-sound signalare input to the processor unit 12 or after the fingerprint image andthe vibration-sound signal are input to the processor unit 12.

In an embodiment, after normalization, during data fusion, the processorunit 12 may crop out a strip-shaped region with a relatively small sizeat an edge of the fingerprint image, and stitch a vibration-sound signalthat has a same size as the strip-shaped region to the edge of thecropped fingerprint image, or may directly stitch a strip-shapedvibration-sound signal to an edge of the fingerprint image. As shown inFIG. 2 , an image size of a fingerprint image may be 160 pixels×160pixels, and an image size of a to-be-determined sample pair obtained bystitching the fingerprint image and a vibration-sound signal may be 172pixels×160 pixels.

Further, the storage unit 15 may store a computer program and thefingerprint anti-counterfeiting model. The fingerprintanti-counterfeiting model is the multi-dimensional network modelobtained through fusion learning or separate learning based on theplurality of fingerprint images for training and the correspondingvibration-sound signals in the multi-dimensional anti-counterfeitingnetwork. The computer program includes instructions, and when theinstructions are executed by the processor, the electronic deviceperforms the following steps:

invoking the fingerprint sensor and the vibration-sound sensor tocollect the fingerprint image and the vibration-sound signal; and

determining, based on the collected fingerprint image, the collectedvibration-sound signal, and the fingerprint anti-counterfeiting model,whether the fingerprint input action is generated by a true finger orgenerated by a fake finger.

The fingerprint anti-counterfeiting model may be prestored in a storageapparatus that can be invoked by the processor unit 12 of the electronicdevice 10. The processor unit 12 may invoke the fingerprintanti-counterfeiting model according to the instructions of the computerprogram, and determine, based on the fingerprint anti-counterfeitingmodel, whether the fingerprint image and the vibration-sound signal aregenerated by the fingerprint input action of the true finger. It may beunderstood that the fingerprint anti-counterfeiting model may be updatedor upgraded. For example, a latest network model is automaticallydownloaded or updated to by using a related network platform, toimplement fast upgraded protection against continuously emerging fakefingerprint attack manners. This has relatively high reliability.

FIG. 3 is a schematic diagram of a principle of a fingerprintanti-counterfeiting model according to an embodiment of thisapplication. The following specifically describes a method for obtaininga fingerprint anti-counterfeiting model and performing fingerprintanti-counterfeiting by using the fingerprint anti-counterfeiting model.First, an electronic device (for example, the electronic device havingthe structure shown in FIG. 1 ) having a fingerprint sensor and avibration-sound sensor may be provided to collect a fingerprint image ofeach press and a vibration-sound signal corresponding to the fingerprintimage. Further, a computer is used to form a training sample pair byusing the collected fingerprint image of the press and the correspondingvibration-sound signal. The sample pair may be a true fingerprinttraining sample pair, or may be a fake fingerprint training sample pair.Further, a large quantity of true or fake fingerprint training samplepairs form a training set of a multi-dimensional network. It may beunderstood that the computer may perform preprocessing (such as datanormalization and fusion), cleaning (such as deleting invalid data), orthe like on the fingerprint image and the corresponding vibration-soundsignal. Then, a device (such as a server) having relatively highcomputing power may be further used to sequentially input trainingsample pairs including fingerprint images and vibration-sound signals inthe training set into a first multi-dimensional anti-counterfeitingnetwork. After repeated learning, the first multi-dimensionalanti-counterfeiting network may obtain a trained fingerprintanti-counterfeiting model. The fingerprint anti-counterfeiting model maybe stored in the storage unit 15 of the electronic device 10, and mayfurther be invoked by the processor unit 12.

In the method shown in FIG. 3 , the first multi-dimensionalanti-counterfeiting network may include a fingerprint image subnetworkand a vibration-sound signal subnetwork. After the training samplesincluding the fingerprint images and the vibration-sound signals in thetraining set enter the first multi-dimensional anti-counterfeitingnetwork, a fingerprint image in each training sample may be separatedfrom a vibration-sound signal, and the fingerprint image and thevibration-sound signal are separately input into the correspondingsubnetworks. After the fingerprint image subnetwork performs featurelearning on the fingerprint image, the fingerprint image subnetwork mayoutput a learned first feature vector related to a fingerprint. Afterthe vibration-sound signal subnetwork performs feature learning on thevibration-sound signal, the vibration-sound signal subnetwork may outputa learned second feature vector related to the vibration-sound signal.The feature vector related to the fingerprint image and the featurevector related to the vibration-sound signal may be fused into a thirdfeature vector. After classification calculation and loss functioncalculation are performed at a fully connected layer, backpropagation isperformed to update a network parameter. A group of network parametersin an optimal state, namely, the fingerprint anti-counterfeiting model,is obtained through repeated learning. It may be understood that, inthis embodiment, the fingerprint anti-counterfeiting model is amulti-dimensional network model obtained through fusion learning basedon a plurality of fingerprint images for training and correspondingvibration-sound signals in the first multi-dimensionalanti-counterfeiting network. The fingerprint image subnetwork is trainedbased on a fingerprint image training set including the plurality offingerprint images for training, and the vibration-sound signalsubnetwork is trained based on a vibration-sound signal training setincluding the plurality of vibration-sound signals corresponding to theplurality of fingerprint images for training.

In the first multi-dimensional anti-counterfeiting network, thefingerprint image subnetwork includes a convolutional neural network,and the vibration-sound signal subnetwork includes a recurrent neuralnetwork. Specifically, it may be understood that a network model havingoptimal performance in a test set may be stored as the trained firstmulti-dimensional network model. In addition, in a model trainingprocess, parameter data of the fingerprint image subnetwork and thevibration-sound signal subnetwork may be synchronously updated, andmodel data having optimal performance in the test set is stored.However, because structures and training data of the two subnetworks aregreatly different, the parameter data of the two subnetworks mayalternatively be asynchronously updated.

Further, in the embodiment shown in FIG. 3 , specifically, the processorunit 12 may input the fingerprint image into the fingerprint imagesubnetwork to obtain the first feature vector, input the vibration-soundsignal into the vibration-sound signal subnetwork to obtain the secondfeature vector, fuse the first feature vector and the second featurevector to obtain the third feature vector, calculate a classificationresult of the third feature vector, and determine, based on theclassification result, that the fingerprint input action is generated bya true finger or generated by a fake finger. The classification resultincludes a confidence level. It may be understood that the confidencelevel is a confidence level that the fingerprint input action isgenerated by the true finger. If the confidence level is greater than orequal to a specified threshold, the processor unit 12 of the electronicdevice 10 determines that the fingerprint input action is generated bythe true finger. If the confidence level is less than the specifiedthreshold, the processor unit 12 of the electronic device 10 determinesthat the fingerprint input action is generated by the fake finger. Forexample, if the confidence level is greater than or equal to thespecified threshold, the processor unit 12 determines that thefingerprint input action is generated by the true finger; or if theconfidence level is less than the specified threshold, the processorunit 12 determines that the fingerprint input action is generated by thefake finger. The confidence level is the confidence level that thefingerprint input action is generated by the true finger. For example,when the classification result is that the confidence level that thefingerprint input action is generated by the true finger is 0.9, and thespecified threshold is 0.5, it may be determined that the fingerprintinput action is generated by the true finger.

FIG. 4 is a schematic diagram of a principle of obtaining a fingerprintanti-counterfeiting model according to an embodiment of thisapplication. The following specifically describes another method forobtaining a fingerprint anti-counterfeiting model. First, an electronicdevice (for example, the electronic device having the structure shown inFIG. 1 ) having a fingerprint sensor and a vibration-sound sensor may beprovided to collect a fingerprint image of each press and avibration-sound signal corresponding to the fingerprint image. Further,a computer is used to form a training sample pair by using the collectedfingerprint image of the press and the corresponding vibration-soundsignal. The sample pair may be a true fingerprint training sample pair,or may be a fake fingerprint training sample pair. Further, a largequantity of true or fake fingerprint training sample pairs form atraining set of a multi-dimensional network. It may be understood thatthe computer may perform preprocessing (such as data normalization andfusion), cleaning (such as deleting invalid data), or the like on thefingerprint image and the corresponding vibration-sound signal. Then, adevice (such as a server) having relatively high computing power may befurther used to sequentially input training sample pairs includingprocessed fingerprint images and vibration-sound signals in the trainingset into a second multi-dimensional anti-counterfeiting network. Thesecond multi-dimensional anti-counterfeiting network includes afingerprint image subnetwork and a vibration-sound signal subnetwork.After repeated learning of the fingerprint images and thevibration-sound signals, the second multi-dimensionalanti-counterfeiting network may obtain a trained fingerprintanti-counterfeiting model. The fingerprint anti-counterfeiting model maybe stored in the storage unit 15 of the electronic device 10, and mayfurther be invoked by the processor unit 12. It may be understood that,in the embodiment shown in FIG. 4 , the fingerprint anti-counterfeitingmodel is a multi-dimensional network model obtained through separatelearning based on a plurality of fingerprint images for training andcorresponding vibration-sound signals in the fingerprint imagesubnetwork and the vibration-sound signal subnetwork.

In the method shown in FIG. 4 , after the training samples including thefingerprint images and the vibration-sound signals in the training setenter the second multi-dimensional anti-counterfeiting network, afingerprint image in each training sample may be separated from avibration-sound signal, and the fingerprint image and thevibration-sound signal are separately input into the correspondingsubnetworks. After the fingerprint image subnetwork performs repeatedfeature learning on data of the input fingerprint image, a group ofparameters having an optimal test result is selected as a fingerprintimage subnetwork. After the vibration-sound signal subnetwork performsrepeated feature learning on the input vibration-sound signal, a groupof parameters having an optimal test result is selected as avibration-sound signal subnetwork. Because the fingerprint imagesubnetwork and the vibration-sound signal subnetwork are obtainedthrough independent training, and are separately stored in theelectronic device or another external storage apparatus that can beinvoked by the processor unit, updating of parameter data of the twosubnetworks in a model training process may not be considered currently.In addition, in the second multi-dimensional anti-counterfeitingnetwork, the fingerprint image subnetwork may also include aconvolutional neural network, and the vibration-sound signal subnetworkalso includes a recurrent neural network.

In the embodiment shown in FIG. 4 , specifically, the processor unit 12may input the fingerprint image into the fingerprint image subnetwork toobtain a first feature vector, input the vibration-sound signal into thevibration-sound signal subnetwork to obtain a second feature vector,calculate a classification result of the first feature vector and thesecond feature vector, and determine, based on the classificationresult, that the fingerprint input action is generated by a true fingeror generated by a fake finger. The classification result includes aconfidence level. It may be understood that the confidence level is aconfidence level that the fingerprint input action is generated by thetrue finger. If the confidence level is greater than or equal to aspecified threshold, the processor unit 12 of the electronic device 10determines that the fingerprint input action is generated by the truefinger. If the confidence level is less than the specified threshold,the processor unit 12 of the electronic device 10 determines that thefingerprint input action is generated by the fake finger. For example,if the confidence level is greater than or equal to the specifiedthreshold, the processor unit 12 determines that the fingerprint inputaction is generated by the true finger; or if the confidence level isless than the specified threshold, the processor unit 12 determines thatthe fingerprint input action is generated by the fake finger. Theconfidence level is the confidence level that the fingerprint inputaction is generated by the true finger. For example, when theclassification result is that the confidence level that the fingerprintinput action is generated by the true finger is 0.9, and the specifiedthreshold is 0.5, it may be determined that the fingerprint input actionis generated by the true finger.

FIG. 5A and FIG. 5B are a flowchart of fingerprint anti-counterfeitingand recognition of the electronic device 10 according to an embodiment.It may be understood that the flowchart shown in FIG. 5A and FIG. 5Bincludes two aspects: fingerprint anti-counterfeiting and fingerprintrecognition. Specifically, when starting to perform fingerprintanti-counterfeiting and recognition, the electronic device 10 mayperform step S1 to first determine whether a trigger event is detected,for example, whether a touch signal is received in a fingerprintrecognition region, and perform steps S2 and S4 if a determining resultis that the trigger event is detected. For example, step S2 is performedto trigger a vibration-sound excitation source to generate avibration-sound excitation signal, and then step S3 is performed totrigger a vibration-sound sensor to collect a vibration-sound signal.Step S4 is performed to trigger a fingerprint chip of a fingerprintsensor to start fingerprint image collection. Further, step S5 and stepS7 are performed. In step S5, the fingerprint sensor obtains afingerprint image, and the processor unit 12 may further perform step S6to perform recognition matching on the fingerprint image. If arecognition matching result is that matching fails, a nextto-be-determined sample pair may be obtained. In other words, steps S2and S4 are performed. In step S7, the fingerprint image obtained in stepS3 and the fingerprint image obtained in step S5 may further benormalized, so that the fingerprint image and the vibration-sound signalare fused to form a to-be-determined sample pair. Then step S8 isperformed to determine, based on a fingerprint anti-counterfeitingmodel, whether the fingerprint image and the vibration-sound signal inthe to-be-determined sample are true or fake. If it is determined thatthe fingerprint image and the vibration-sound signal are generated by afingerprint input action of a true finger and the recognition matchingresult in step S6 is that matching succeeds, step S9 is performed tooutput an identity authentication result indicating that the fingerprintrecognition succeeds. If it is determined that the fingerprint image andthe vibration-sound signal are generated by a fingerprint input actionof a fake finger or the recognition matching result in step S6 is thatmatching fails, step S10 is performed to output an identityauthentication result indicating that the fingerprint recognition fails.

It may be understood that, the foregoing descriptions are mainly for anelectronic device having a vibration-sound excitation source. When thedetermining result in step S1 is that the trigger event is detected, theelectronic device performs step S2. But for an electronic device havingno vibration-sound excitation source 112 a, when the determining resultin step S1 is that the trigger event is detected, the electronic devicemay alternatively directly perform step S3. In addition, for afingerprint image and a vibration-sound signal on which neither ofnormalization and data fusion needs to be performed, the fingerprintimage and the vibration-sound signal on which neither of normalizationand fusion is performed may be separately input into the fingerprintanti-counterfeiting model. The fingerprint anti-counterfeiting model isused to separately perform determining on the fingerprint image and thevibration-sound signal. If it is determined that the fingerprint imageand the vibration-sound signal are generated by a fingerprint inputaction of a true finger, step S9 is performed to output an identityauthentication success result. If it is determined that the fingerprintimage and the vibration-sound signal are generated by a fingerprintinput action of a fake finger, step S10 is performed to output anidentity authentication failure result.

In addition, it may be understood that, when the electronic device 10determines that the fingerprint input action is generated by the fakefinger, the method further includes: The electronic device 10 determinesthat the fingerprint recognition fails, and sends prompt informationindicating that the fingerprint recognition fails.

Optionally, the fingerprint recognition procedure in step S6 and thefingerprint anti-counterfeiting procedure in step S8 may be performed insequence, or the fingerprint anti-counterfeiting procedure in step S8may be performed before or after the fingerprint recognition procedurein step S6. However, in the embodiment shown in FIG. 5A and FIG. 5B, thetwo procedures are performed in parallel.

FIG. 6 is a flowchart of a fingerprint anti-counterfeiting methodaccording to an embodiment of this application. The foregoing describesthe principle for the electronic device 10 in this application toperform fingerprint anti-counterfeiting based on the vibration-soundsignal. The following briefly describes, with reference to FIG. 5A andFIG. 5B, how the processor unit 12 of the electronic device 10 performsthe fingerprint anti-counterfeiting method. The fingerprintanti-counterfeiting method may include the following steps: step S61,step S62, step S63, and step S64.

Step S61: Obtain a fingerprint image generated by a fingerprint inputaction.

Step S62: Obtain a vibration-sound signal generated by the fingerprintinput action.

Step S63: Determine, based on a fingerprint anti-counterfeiting model,whether the fingerprint image and the vibration-sound signal aregenerated by a fingerprint input action of a true finger, where thefingerprint anti-counterfeiting model is a multi-dimensional networkmodel obtained through fusion learning or separate learning based on aplurality of fingerprint images for training and correspondingvibration-sound signals in a multi-dimensional anti-counterfeitingnetwork.

Specifically, in an embodiment, as shown in FIG. 5A and FIG. 5B, themethod may further include: step S1 that is of detecting whether atrigger event occurs and that is performed before step S61, step S4 ofcontrolling, when a detection result is that the trigger event occurs,the fingerprint sensor 111 to start fingerprint image collection toobtain the fingerprint image generated by the fingerprint input action,and step S3 of controlling the vibration-sound sensor 112 to startvibration and sound collection to obtain the vibration-sound signalgenerated by the fingerprint input action.

Specifically, in another embodiment, as shown in FIG. 6 , the methodfurther includes: detecting whether a trigger event occurs in step S1,and when a detection result is that the trigger event occurs,controlling the fingerprint sensor 111 to start fingerprint imagecollection to obtain the fingerprint image generated by the fingerprintinput action, controlling a vibration-sound excitation source to emit avibration-sound excitation signal, and controlling the vibration-soundsensor 112 to start vibration and sound collection after a presetlatency that starts to be calculated when the vibration-sound excitationsignal is emitted, to obtain the vibration-sound signal generated by thefingerprint input action.

Further, as described above, the vibration-sound signal includes amechanical vibration signal and a sound signal, and the vibration-soundsensor 112 includes at least one of a microphone, a sound sensor, anacceleration sensor, a crash sensor, a displacement sensor, theacceleration sensor, and a force sensor. The fingerprint sensor 111includes at least one of an under-display/in-display optical fingerprintsensor, an ultrasonic fingerprint sensor, and a capacitive fingerprintsensor.

Further, as described above, the fingerprint anti-counterfeiting modelmay include a network model obtained through training based on aplurality of training sample pairs including the plurality offingerprint images for training and the corresponding vibration-soundsignals in the multi-dimensional anti-counterfeiting network (forexample, a first multi-dimensional anti-counterfeiting network), andstep S64 may include the following steps:

forming a to-be-determined sample pair by using the fingerprint imageand the vibration-sound signal; and

inputting the to-be-determined sample pair into the fingerprintanti-counterfeiting model to obtain a computation result.

Further, as shown in FIG. 5A and FIG. 5B, the method may further includestep S7 of separately normalizing the fingerprint image and thevibration-sound signal before the step of forming the to-be-determinedsample pair by using the fingerprint image and the vibration-soundsignal.

Further, in an embodiment, the multi-dimensional anti-counterfeitingnetwork (such as the first multi-dimensional anti-counterfeitingnetwork) includes a fingerprint image subnetwork and a vibration-soundsignal subnetwork. The fingerprint image subnetwork and thevibration-sound signal subnetwork are respectively used to performfeature extraction on the fingerprint image and the vibration-soundsignal. The fingerprint image subnetwork is trained based on afingerprint image training set including the plurality of fingerprintimages for training, and the vibration-sound signal subnetwork istrained based on a vibration-sound signal training set including theplurality of vibration-sound signals corresponding to the plurality offingerprint images for training. The step of the determining, based on afingerprint anti-counterfeiting model, whether the fingerprint image andthe vibration-sound signal are generated by a fingerprint input actionof a true finger includes:

inputting the fingerprint image into the fingerprint image subnetwork toobtain a first feature vector;

inputting the vibration-sound signal into the vibration-sound signalsubnetwork to obtain a second feature vector;

fusing the first feature vector and the second feature vector to obtaina third feature vector; and

inputting, by the electronic device 10, the third feature vector into aclassification layer (for example, a softmax function, a sigmoidfunction, or another function having a classification effect) of themulti-dimensional anti-counterfeiting network for classification tocalculate a classification result.

In another embodiment, the multi-dimensional anti-counterfeiting networkincludes a second multi-dimensional anti-counterfeiting network,including a fingerprint image subnetwork and a vibration-sound signalsubnetwork. The fingerprint image subnetwork is trained based on afingerprint image training set including the plurality of fingerprintimages for training, and the vibration-sound signal subnetwork istrained based on a vibration-sound signal training set including theplurality of vibration-sound signals corresponding to the plurality offingerprint images for training. The step of the determining, based on afingerprint anti-counterfeiting model, whether the fingerprint image andthe vibration-sound signal are generated by a fingerprint input actionof a true finger includes:

inputting the fingerprint image into the fingerprint image subnetwork toobtain a first feature vector;

inputting the vibration-sound signal into the vibration-sound signalsubnetwork to obtain a second feature vector;

classifying, by the electronic device, the first feature vector and thesecond feature vector to calculate a classification result; and

determining, by the electronic device based on the classificationresult, that the fingerprint input action is generated by the truefinger or generated by a fake finger.

In the another embodiment, it may be understood that the fingerprintimage and the vibration-sound signal may also be normalized, and thenthe normalized fingerprint image and vibration-sound signal areseparately input into the second multi-dimensional anti-counterfeitingnetwork.

As described above, in the foregoing two embodiments, that theelectronic device 10 determines, based on the classification result,that the fingerprint input action is generated by the true finger orgenerated by the fake finger may include:

When the classification result includes a confidence level, if theconfidence level is greater than or equal to a specified threshold, theelectronic device 10 (for example, the processor unit 12) determinesthat the fingerprint input action is generated by the true finger; or ifthe confidence level is less than the specified threshold, theelectronic device 10 determines that the fingerprint input action isgenerated by the fake finger. The confidence level is a possibility thatthe fingerprint input action is generated by the true finger.

For example, a result obtained through calculation is the confidencelevel that the fingerprint input action is generated by the true finger.The electronic device determines, based on the classification result,that the fingerprint input action is generated by the true finger orgenerated by the fake finger. For example, when the classificationresult is that the confidence level that the fingerprint input action isgenerated by the true finger is 0.9, and the specified threshold is 0.5,it may be determined that the fingerprint input action is generated bythe true finger.

Further, as described above, in the foregoing embodiments, beforeinputting the fingerprint image and the vibration-sound signal into themulti-dimensional anti-counterfeiting network, the electronic device 10may fuse the fingerprint image and the vibration-sound signal.

In addition, after inputting the fingerprint image and thevibration-sound signal into the multi-dimensional anti-counterfeitingnetwork, the electronic device 10 may separate the fingerprint imagefrom the vibration-sound signal.

It may be understood that step S63 may include the foregoing steps S8,S9, and S10. As shown in steps S8, S9, and S10, when the electronicdevice 10 determines that the fingerprint input action is generated bythe fake finger, the method further includes: The electronic device 10determines that fingerprint recognition fails, and sends promptinformation indicating that the fingerprint recognition fails.

Further, when the electronic device 10 determines that the fingerprintinput action is generated by the true finger, the method furtherincludes: The electronic device 10 determines whether the fingerprintimage matches a preset fingerprint image. If the fingerprint imagematches the preset fingerprint image, the electronic device 10determines that fingerprint unlock succeeds. Optionally, the fingerprintrecognition procedure in step S6 and the fingerprint anti-counterfeitingprocedure in step S8 may be performed in parallel, or the fingerprintanti-counterfeiting procedure in step S8 may be performed before orafter the fingerprint recognition procedure in step S6. However, asshown in FIG. 5A and FIG. 5B, preferably, the two procedures areperformed in parallel.

Content such as the first multi-dimensional anti-counterfeiting network,the second multi-dimensional anti-counterfeiting network, and how toobtain the fingerprint anti-counterfeiting model is described in detailabove.

Compared with the conventional technology, in the fingerprintanti-counterfeiting method and the electronic device 10 in thisapplication, determining, based on the vibration-sound signal, whetherthe fingerprint image is true or fake may effectively defend against afake fingerprint attack (especially a 3D fake fingerprint attack) facedby a current fingerprint solution. In addition, a structure and anintegrated application of the vibration-sound sensor 112 are alsorelatively simple, and therefore, a cost increase and integrationdifficulty caused by a hardware-based anti-counterfeiting solution canbe effectively avoided. In addition, because the fingerprintanti-counterfeiting model (for example, the first multi-dimensionalnetwork model and the second multi-dimensional network model) used forfingerprint anti-counterfeiting may provide fast upgraded protectionagainst continuously emerging fake fingerprint attack manners,reliability is relatively high.

In conclusion, the foregoing embodiments are merely intended fordescribing the technical solutions of this application, but not forlimiting this application. Although this application is described indetail with reference to the foregoing embodiments, persons of ordinaryskill in the art should understand that they may still makemodifications to the technical solutions described in the foregoingembodiments or make equivalent replacements to some technical featuresthereof, without departing from the scope of the technical solutions ofembodiments of this application. FIG. 7 is a schematic diagram of astructure of an electronic apparatus 7 for performing the method in theforegoing embodiment. The electronic apparatus 7 includes but is notlimited to at least one memory 71, at least one processor 72, at leastone communications apparatus 73, and at least one communications bus.The communications bus is configured to implement connection andcommunication between these components.

The electronic apparatus 7 is a device that can automatically performvalue calculation and/or information processing according to preset orstored instructions. Hardware of the electronic apparatus 7 includes butis not limited to a microprocessor, an application-specific integratedcircuit (ASIC), a field-programmable gate array (FPGA), and a digitalsignal processor (DSP), an embedded device, and the like. The electronicapparatus 7 may further include a network device and/or user equipment.The network device includes but is not limited to a single networkserver, a server group including a plurality of network servers, or acloud including a large quantity of hosts or network servers based oncloud computing, where the cloud computing is a type of distributedcomputing, and the cloud is a super virtual computer including a groupof loosely coupled computers.

The electronic apparatus 7 may be but is not limited to any electronicproduct, for example, a terminal such as a tablet computer, asmartphone, a personal digital assistant (PDA), an intelligent wearabledevice, a camera device, or a monitoring device, that can performman-machine interaction with a user by using a keyboard, a touchpad, avoice control device, or the like.

A network in which the electronic apparatus 7 is located includes but isnot limited to the Internet, a wide area network, a metropolitan areanetwork, a local area network, a virtual private network (VPN), or thelike.

The communications apparatus may be a wired sending port, or may be awireless device. For example, the communications apparatus includes anantenna apparatus, configured to perform data communication with anotherdevice.

The memory 71 is configured to store program code. The memory 71 may bea circuit having a storage function and having no physical form in anintegrated circuit. For example, the memory 71 may be a RAM (RandomAccess Memory) or a FIFO (First In First Out) memory. Alternatively, thememory may be a memory having a physical form, for example, a storagedevice such as a memory module, a TF card (Trans-flash Card), a smartmedia card, a secure digital card, or a flash memory card.

The processor 72 may include one or more microprocessors and digitalprocessors. The processor 52 may invoke the program code stored in thememory to perform a related function. For example, the modules in FIG. 6are program code stored in the memory 71 and are executed by theprocessor 72, to implement the fingerprint anti-counterfeiting method.The processor is also referred to as a central processing unit (CPU), isa very large-scale integrated circuit, and is a computing core and acontrol core (Control Unit).

An embodiment of this application further provides a computer-readablestorage medium. The computer-readable storage medium stores computerinstructions. When the instructions are executed by one or moreprocessors, the fingerprint anti-counterfeiting method in the foregoingmethod embodiment is performed.

The characteristic means in this application described above may beimplemented by using an integrated circuit, and a function of thefingerprint anti-counterfeiting method in any one of the foregoingembodiments is controlled to be implemented.

All functions that can be implemented by the fingerprintanti-counterfeiting method in any embodiment can be installed in theelectronic device by using the integrated circuit in this application,so that the electronic device performs the functions that can beimplemented by the fingerprint anti-counterfeiting method in anyembodiment. Details are not described herein again.

It should be noted that for brief description, the foregoing methodembodiments are described as a series of actions. However, personsskilled in the art should appreciate that this application is notlimited to the described order of the actions, because according to thisapplication, some steps may be performed in another sequence orsimultaneously. It should be further appreciated by persons skilled inthe art that embodiments described in this specification all belong topreferred embodiments, and the involved actions and modules are notnecessarily required by this application.

In the foregoing embodiments, descriptions of embodiments haverespective focuses.

For a part that is not described in detail in an embodiment, refer torelated descriptions in other embodiments.

In several embodiments provided in this application, it should beunderstood that the disclosed apparatuses may be implemented in othermanners. For example, the described apparatus embodiments are merelyexamples. For example, division into the units is merely logicalfunction division and may be other division in an actual implementation.For example, a plurality of units or components may be combined orintegrated into another system, or some features may be ignored or notperformed. In addition, the displayed or discussed mutual couplings ordirect couplings or communication connections may be implemented throughsome interfaces. The indirect couplings or communication connectionsbetween the apparatuses or units may be implemented in electrical formor other forms.

The units described as separate parts may or may not be physicallyseparate, and parts displayed as units may or may not be physical units,and may be located in one position, or may be distributed on a pluralityof network units. Some or all of the units may be selected according toactual requirements to achieve the objectives of the solutions ofembodiments.

In addition, function units in embodiments of this application may beintegrated into one processing unit, or each of the units may existalone physically, or two or more units may be integrated into one unit.The integrated unit may be implemented in a form of hardware, or may beimplemented in a form of a software function unit.

When the integrated unit is implemented in the form of the softwarefunction unit and sold or used as an independent product, the integratedunit may be stored in a computer-readable storage medium. Based on suchan understanding, the technical solutions of this applicationessentially, or the part contributing to the conventional technology, orall or some of the technical solutions may be implemented in a form of asoftware product. The computer software product is stored in a storagemedium and includes several instructions for instructing a computerdevice (which may be a personal computer, a server, a network device, orthe like) to perform all or some of the steps of the methods in theembodiments of this application. The foregoing storage medium includesany medium that can store program code, such as a USB flash drive, aread-only memory (ROM), a random access memory (RAM), a removable harddisk, a magnetic disk, or an optical disc.

In conclusion, the foregoing embodiments are merely intended fordescribing the technical solutions of this application, but not forlimiting this application. Although this application is described indetail with reference to the foregoing embodiments, persons of ordinaryskill in the art should understand that they may still makemodifications to the technical solutions described in the foregoingembodiments or make equivalent replacements to some technical featuresthereof, without departing from the scope of the technical solutions ofembodiments of this application.

1. A fingerprint anti-counterfeiting method performed by an electronicdevice, comprising: obtaining a fingerprint image generated by afingerprint input action; obtaining a vibration-sound signal generatedby the fingerprint input action; and determining, based on a fingerprintanti-counterfeiting model, whether the fingerprint input action isperformed by a true finger, wherein the fingerprint anti-counterfeitingmodel is a multi-dimensional network model obtained through fusionlearning or separate learning based on a plurality of fingerprint imagesfor training and corresponding vibration-sound signals in amulti-dimensional anti-counterfeiting network.
 2. The method accordingto claim 1, wherein the fingerprint anti-counterfeiting model comprisesa multi-dimensional network model obtained through training based on aplurality of training sample pairs comprising the plurality offingerprint images for training and the corresponding vibration-soundsignals in the multi-dimensional anti-counterfeiting network, anddetermining whether the fingerprint input action is performed by thetrue finger comprises: forming a to-be-determined sample pair by usingthe fingerprint image and the vibration-sound signal; and inputting theto-be-determined sample pair into the multi-dimensional network model toobtain a computation result.
 3. (canceled)
 4. The method according toclaim 2, wherein the multi-dimensional anti-counterfeiting networkcomprises a fingerprint image subnetwork and a vibration-sound signalsubnetwork respectively for performing feature extraction on thefingerprint image and the vibration-sound signal; wherein the pluralityof fingerprint images for training and the corresponding vibration-soundsignals are separately normalized before the plurality of fingerprintimages for training and the corresponding vibration-sound signals formthe plurality of training sample pairs, and the method further comprisesseparately normalizing the fingerprint image and the vibration-soundsignal before forming the to-be-determined sample pair by using thefingerprint image and the vibration-sound signal. 5-9. (canceled) 10.The method according to claim 1, further comprising: detecting whether atrigger event occurs, and upon detecting that the trigger event occurs,controlling a fingerprint sensor to start fingerprint image collectionto obtain the fingerprint image generated by the fingerprint inputaction, and controlling a vibration-sound sensor to start vibration andsound collection to obtain the vibration-sound signal generated by thefingerprint input action.
 11. The method according to claim 1, furthercomprising: detecting whether a trigger event occurs, and upon detectingthat the trigger event occurs, controlling a fingerprint sensor to startcollecting the fingerprint image generated by the fingerprint inputaction, controlling a vibration-sound excitation source to emit avibration-sound excitation signal, and controlling a vibration-soundsensor to start collecting the vibration-sound signal generated by thefingerprint input action after a preset latency that starts to becalculated when the vibration-sound excitation signal is emitted. 12.The method according to claim 1, wherein when the electronic devicedetermines that the fingerprint input action is generated by a fakefinger, the method further comprises: determining, by the electronicdevice, that fingerprint recognition fails, and sending promptinformation indicating that the fingerprint recognition fails.
 13. Themethod according to claim 1, wherein when the electronic devicedetermines that the fingerprint input action is generated by the truefinger, the method further comprises: determining, by the electronicdevice, whether the fingerprint image matches a preset fingerprintimage; and when the fingerprint image matches the preset fingerprintimage, determining, by the electronic device, that fingerprintrecognition succeeds.
 14. The method according to claim 10, wherein thevibration-sound signal comprises a mechanical vibration signal and asound signal, and the vibration-sound sensor comprises at least one of amicrophone, a sound sensor, an acceleration sensor, a crash sensor, adisplacement sensor, the acceleration sensor, or a force sensor.
 15. Themethod according to claim 10, wherein the fingerprint sensor comprisesat least one of an under-display/in-display optical fingerprint sensor,an ultrasonic fingerprint sensor, or a capacitive fingerprint sensor.16. An electronic device, comprising an input, a processor, and astorage, wherein the input comprises a fingerprint sensor and avibration-sound sensor, the fingerprint sensor is configured to collecta fingerprint image of a fingerprint input action, and thevibration-sound sensor is configured to collect a vibration-sound signalgenerated by the fingerprint input action; and the storage stores acomputer program and a fingerprint anti-counterfeiting model, whereinthe fingerprint anti-counterfeiting model is a multi-dimensional networkmodel obtained through fusion learning or separate learning based on aplurality of fingerprint images for training and a plurality ofcorresponding vibration-sound signals in a multi-dimensionalanti-counterfeiting network, the computer program comprisesinstructions, and when the instructions are executed by the processor,the electronic device performs operations comprising: invoking thefingerprint sensor and the vibration-sound sensor to collect thefingerprint image and the vibration-sound signal; and determining, basedon the collected fingerprint image, the collected vibration-soundsignal, and the fingerprint anti-counterfeiting model, whether thefingerprint input action is generated by a true finger or generated by afake finger.
 17. The electronic device according to claim 16, whereinthe vibration-sound sensor comprises a vibration-sound excitation sourceand a vibration-sound sensing module, the vibration-sound excitationsource is configured to emit a vibration-sound excitation signal, andthe vibration-sound sensing module is configured to collect thevibration-sound signal generated by the fingerprint input action. 18.The electronic device according to claim 17, wherein the vibration-soundexcitation source is configured to emit the vibration-sound excitationsignal when being triggered by the fingerprint input action, and thevibration-sound sensing module is configured to start, after a presetlatency that starts to be calculated when the vibration-sound excitationsource is triggered, detecting the vibration-sound signal generated byboth the fingerprint input action and the vibration-sound excitationsignal.
 19. The electronic device according to claim 16, wherein thevibration-sound signal comprises a mechanical vibration signal and asound signal, and the vibration-sound sensor comprises at least one of amicrophone, a sound sensor, an acceleration sensor, a crash sensor, adisplacement sensor, the acceleration sensor, or a force sensor.
 20. Theelectronic device according to claim 16, wherein the fingerprint sensorcomprises at least one of an under-display/in-display opticalfingerprint sensor, an ultrasonic fingerprint sensor, or a capacitivefingerprint sensor.
 21. The electronic device according to claim 16,wherein the fingerprint anti-counterfeiting model comprises a modelobtained through training based on a plurality of training sample pairscomprising the plurality of fingerprint images for training and thecorresponding vibration-sound signals, and the processor is furtherconfigured to: form a to-be-determined sample pair by using thefingerprint image and the vibration-sound signal, input theto-be-determined sample pair into the fingerprint anti-counterfeitingmodel to obtain a confidence level indicating a probability that thefingerprint input action is a true fingerprint, and determine, based onthe confidence level and a specified threshold, whether the fingerprintinput action is generated by the true finger.
 22. The electronic deviceaccording to claim 16, wherein the multi-dimensional anti-counterfeitingnetwork comprises a fingerprint image subnetwork and a vibration-soundsignal subnetwork.
 23. The electronic device according to claim 22,wherein the fingerprint image subnetwork is obtained through trainingbased on a fingerprint image training set comprising the plurality offingerprint images for training, the vibration-sound signal subnetworkis obtained through training based on a vibration-sound signal trainingset comprising the plurality of vibration-sound signals corresponding tothe plurality of fingerprint images for training, and the processor isfurther configured to: input the fingerprint image into the fingerprintimage subnetwork to obtain a first feature vector; input thevibration-sound signal into the vibration-sound signal subnetwork toobtain a second feature vector; fuse the first feature vector and thesecond feature vector to obtain a third feature vector; classify thethird feature vector to calculate a classification result; anddetermine, based on the classification result, that the fingerprintinput action is generated by the true finger or generated by the fakefinger.
 24. The electronic device according to claim 22, wherein thefingerprint image subnetwork is obtained through training based on afingerprint image training set comprising the plurality of fingerprintimages for training, the vibration-sound signal subnetwork is obtainedthrough training based on a vibration-sound signal training setcomprising the plurality of vibration-sound signals corresponding to theplurality of fingerprint images for training, and the processor isfurther configured to: input the fingerprint image into the fingerprintimage subnetwork to obtain a first feature vector; input thevibration-sound signal into the vibration-sound signal subnetwork toobtain a second feature vector; classify the first feature vector andthe second feature vector to calculate a classification result; anddetermine, based on the classification result, that the fingerprintinput action is generated by the true finger or generated by the fakefinger.
 25. The electronic device according to claim 23, wherein thefingerprint image subnetwork comprises a convolutional neural network,and the vibration-sound signal subnetwork comprises a recurrent neuralnetwork.
 26. (canceled)
 27. An electronic apparatus, comprising aprocessor and a memory, wherein the memory stores at least oneinstruction, and when the at least one instruction is executed by theprocessor, the electronic apparatus performs operations comprising:detecting a fingerprint input action; obtaining a fingerprint imagegenerated by the fingerprint input action; obtaining a vibration-soundsignal generated by the fingerprint input action; and determining, basedon a fingerprint anti-counterfeiting model, whether the fingerprintinput action is performed by a true finger, wherein the fingerprintanti-counterfeiting model is a multi-dimensional network model obtainedthrough fusion learning or separate learning based on a plurality offingerprint images for training and corresponding vibration-soundsignals in a multi-dimensional anti-counterfeiting network.